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(57) ; According to the prior art, the determination of 
edge picture elements and that of moire picture ele- 
ments are separately executed, with image data 
retouching executed by still another processing, result- 
ing in complex processing and configuration. 

By acquiring image data from an image input 
device such as a color scanner 20, differences of tone 
levels of picture elements constituting these image data 
from surrounding picture elements are figured out, the 
distribution of these differences of tone levels is pre- 
pared, and integrated processing based on this distribu- 
tion makes *rt possible to determine whether each 
picture element is an edge picture element, a moire pic- 
ture element or an intermediate picture element. Then 
according to the determination, a sharpening filter is 
applied if the picture elements are edge picture ele- 
ments, a smoothing filter is applied if they are moire pic- 
ture elements, or the tones of the original image are 
held if they are intermediate picture elements. This way 
of image data retouching makes it possible to carry out 
the whole execution from determination to retouching in 
a serial process. This facilitates simplification of the flow 
of processing from determination to image data 
retouching, and at the same time speeding up the 
processing. Thus the user of the color copying machine 
10 is enabled to acquire prints of appropriately 



retouched images at higher speed. 
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Description 

Technical Field 

5 [0001] The present invention relates to an image data retouching apparatus, an image data retouching method and 
a medium on which an image data retouching program is recorded, all for executing image data retouching according 
to the characteristics of individual picture elements. 

Background Art 

10 

[0002] In order to improve the quality of images picked up by a scanner or a digital camera, image data on these 
images are subjected to retouching. More specifically, digital data are put into a computer, on which a photo-retouching 
application is actuated to emphasize desired color contents and/or contrast. Or prescribed color contents are removed 
to weaken their influence. These are some of the various retouching options that are available. 

15 [0003] For document images of which the data to be retouched mainly consist of characters or photographic 
images mainly composed of photographic elements, it is sufficient to retouch the image data constituting their principal 
parts, and the required processing is relatively simple. On the other hand, as DTP techniques have become highly 
developed to make possible ready preparation of presentation documents, in which characters and color graphics are 
integrated, or album images, in which characters and photographs are similarly arranged, there emerge frequent needs 

20 for so-called mixed images, in which characters and photographic elements are mixed. 

[0004] Character elements of image data in such mixed images tend to have intermediate tones on their borders 
with the background, and consequently the characters often become blurred and poor in readability. On the other hand, 
photographic elements are subject to periodic noise attributable to high-frequency components, and this noise some- 
times invites visually perceivable deterioration of picture quality. 

25 [0005] Accordingly, when image data in mixed images are to be retouched, the contrast on borders between char- 
acters and their background is emphasized to clear the blurring of character elements. This can serve to enhance the 
readability of the characters. To do this, prescribed sharpening to emphasize contrast is executed on each picture ele- 
ment on the borders. 

[0006] On the other hand for photographic elements, as the aforementioned noise consists of high-frequency com- 
30 ponents, each picture element with noise is subjected to a prescribed smoothing procedure to reduce high-frequency 
components. In this context, border zones of character elements are generally referred to as edge picture elements, 
and the periodic noise attributable high-frequency components arising in photographic elements, as moire picture ele- 
ments. 

[0007] Image data retouching applicable to such mixed images will be outlined below. First, image data of images 

35 picked up by a scanner or a digital camera are acquired as image data which express picture elements in a dot matrix 
form in multiple tones, and picture elements constituting these image data are successively scanned to compute differ- 
ences of tone levels from surrounding picture elements in a prescribed range with reference to a criterion picture ele- 
ment. If there are many picture elements with large differences of tone levels, such elements are extracted as edge 
picture elements. With the positions of these extracted edge picture elements being taken into consideration, the range 

40 of edge picture elements to be sharpened are demarcated and determined. 

[0008] Then, the picture elements constituting the image data are successively scanned again to compute differ- 
ences of tone levels of picture elements in the prescribed range from the surrounding picture elements with reference 
to a criterion picture element, and if there are many picture elements whose differences of tone levels are relatively 
small, they are extracted as moire picture elements. With the positions of these extracted moire picture elements being 

45 taken into consideration, the range of moire picture elements to be smoothed are demarcated and determined. Once 
the range of edge picture elements and that of moire picture elements are determined in this manner, sharpening and 
smoothing are executed. In this case, the sequence is that sharpening is executed first, followed by smoothing. 
[0009] At the sharpening step, the range of edge picture elements is extracted and sharpened, and the sharpened 
range of edge picture elements is held. 

so [0010] At the smoothing step, the range of moire picture elements is extracted and smoothed, and the smoothed 
range of moire picture elements is held. Thus, ranges to be sharpened and ranges to be smoothed, both distinguished 
element by element, are extracted individually, and processed individually. 

[0011] And image data in each range having been appropriately processed and held are relatively positioned for 
integration, and retouched image data are thereby generated. 

55 

Disclosure of the Invention 

[0012] In the above described processing for image data retouching, the determination of edge picture elements 
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and that of moire picture elements are executed in separate procedures, and the range of edge picture elements and 
that of moire picture elements thereby demarcated are subjected to image data retouching, again in separate proce- 
dures. Therefore, this way of processing involves the problem that the procedures to determine edge picture elements 
and moire picture elements and those to retouch the image data of picture elements are complex in configuration. This 
5 inevitable complexity of processing configuration results in slow processing speed and consequent inconvenience for 
the user. 

[0013] The present invention, attempted in view of the above-noted problems, is intended to provide a medium on 
which an image data retouching program is recorded, an image data retouching apparatus and an image data retouch- 
ing method, all capable of simplifying and increasing the speed of processing by efficiently executing the determination 
w of the characteristic of picture elements, including whether they are edge picture elements or moire picture elements 
and facilitating the retouching of corresponding image data. 

[0014] In order to achieve the above-stated object, according to Claim 1 of the present invention, there is provided 
an image data retouching apparatus for determining the characteristic of each picture element in image data which 
express images in a dot matrix form in multiple tones and retouching each picture element in a way appropriate to its 

is characteristic, the apparatus comprising: an image data acquisition unit for acquiring the image data; a picture element 
characteristic determining unit for figuring out the distribution of differences of tone levels between object picture ele- 
ments, which are the picture elements of the image data acquired by the image data acquisition unit, and neighboring 
picture elements in a prescribed range around the object picture elements, and determining the characteristic of the 
object picture elements by comparing the distribution so figured out and a prescribed model distribution; and an image 

20 data retouching unit for executing prescribed image processing according to the characteristic of picture elements 
determined by the picture element characteristic determining unit. 

[0015] In the apparatus configured as described above according to Claim 1 of the invention, the image data acqui- 
sition unit acquires image data which express images in a dot matrix form in multiple tones. The picture element char- 
acteristic determining unit determines the characteristic of each of the picture elements in the image data. In doing so, 
25 the unit figures out the distribution of differences of tone levels of these picture elements as object picture elements from 
surrounding picture elements in the prescribed range around the object picture elements. The distribution so figured out 
is compared with the prescribed model distribution to determine the characteristics of the object picture elements, and 
the image data retouching unit executes the prescribed image processing according to the determined characteristics 
of the picture elements. 

30 [0016] Thus, instead of altering the way of determination according to the characteristic, the distribution of differ- 
ences of tone levels from neighboring picture elements is figured out and the characteristic of picture elements is deter- 
mined according to the pattern of that distribution. 
' [0017] Here, the picture element characteristic determining unit figures out the distribution of differences of tone 
levels from object picture elements in a prescribed range around the object picture elements, and compares this distri- 

35 bution with the model distribution from a statistical point of view. As one of its conceivable specific example, Claim 2 of 
the invention provides for a picture element characteristic determining unit in the image data retouching apparatus 
described in Claim 1 , uses the distribution in a range where there are less differences of tone levels and that in a range 
where there are more differences of tone levels. 

[0018] In the above-described configuration according to Claim 2 of the invention, the frequency distributions of dff- 
40 ferences of tone levels are figured out, not for individual tone levels but the distribution in a range where there are less 
differences of tone levels and the distribution in a range where there are more differences of tone levels. If the charac- 
teristic of picture element to be determined is the presence or absence of edges, since edges are distributed where dif- 
ferences of tone levels are greater, many edges will necessarily be found distributed in a range where the differences 
are greater. Or if moire is found to have arisen, picture elements with less differences of tone levels must have occurred 
45 in a range of minute differences. Of course a similar trend may be found where there is no edge or moire, but this is no 
more than one of available criteria, and this criterion can be combined with some other technique of determination to 
enhance the precision of determination. Accordingly, this corresponds to one of available model distributions. 
[0019] As another example of picture element characteristic determining unit in this sense, according to Claim 3 of 
the invention, in the above-described image data retouching apparatus according to Claim 2 of the invention, the picture 
so element characteristic determining unit uses the positive or negative polarization of the distribution. 

[0020] The configuration according to Claim 3 of the invention bases the determination on whether the distribution 
of difference of tone levels is polarized, toward the positive range or the negative range. 

[0021] For instance, since an edge is a range where picture elements with large differences of tone levels consec- 
utively occur, in an overall outlook there is a side where differences of tone levels are greater and a side where the dif- 
55 ferences are smaller. If the object picture elements are on the side where differences of tone levels are greater, the 
distribution of the differences will tend to be close to zero and polarize toward the positive side. Or if the object picture 
elements are on the side where differences of tone levels are smaller, the distribution of the differences will tend to be 
close to zero and polarize toward the negative side. Accordingly, positive or negative polarization can be regarded as a 
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model distribution, and utilized as such for the determination. 

[0022] As an example of configuration specifically realizing the above -described way of determination, according to 
Claim 4 of the invention, in the image data retouching apparatus according to Claim 3 of the invention, the picture ele- 
ment characteristic determining unit determines that, if many picture elements with large differences of tone levels are 
5 extensively distributed and this distribution is polarized to either the positive or the negative side, they are edge picture 
elements. 

[0023] In other words, the distribution in the range of large differences of tone levels and the positive or negative 
polarization are compared as the basis of determining given picture elements to be edge picture elements. 
[0024] Further, according to Claim 5 of the invention, in the image data retouching apparatus according to Claim 4 
io of the invention, the image data retouching unit has a configuration to execute sharpening of pictures when the object 
picture elements are determined to be edge picture elements. 

[0025] If the object picture elements are determined to be edge picture elements, the determination is utilized as it 
is to execute sharpening to prevent the edges from being dulled. 

[0026] According to Claim 6 of the invention, in the image data retouching apparatus according to Claim 5 of the 
15 invention, the configuration is such that the sharpening is accomplished by a sharpening filter in which a matrix having 
a prescribed number of picture elements centering on the object picture elements is formed, and a prescribed coeffi- 
cient is set in the position of each picture in the matrix to emphasize that object picture element. 
[0027] As picture elements in a prescribed range centering on the object picture elements are the objects of deter- 
mination, if a sharpening filter to process picture elements in the same prescribed range is used, the determination of 
20 the picture element characteristic and image processing will well match each other, resulting in efficient execution of 
sharpening. 

[0028] Another example of configuration specifically realizing the above- described way of determination, according 
to Claim 7 of the invention, in the image data retouching apparatus according to Claim 3 or 4 of the invention, the picture 
element characteristic determining unit determines that, if many picture elements with small differences of tone levels 
25 are extensively distributed and this distribution is polarized to either the positive or the negative side, they are moire pic- 
ture elements. 

[0029] Thus, by combining the distribution of picture elements with small differences of tone levels and the polari- 
zation of distribution to either the positive or the negative side, the pertinent picture elements are determined to be 
moire picture elements. 

30 [0030] According to Claim 8 of the invention, in the image data retouching apparatus according to Claim 7 of the 
invention, the image data retouching unit has a configuration to execute smoothing of pictures when the object picture 
elements are determined to be moire picture elements. 

[0031] If the object picture elements are determined to be moire picture elements, the determination is utilized as 
it is to execute smoothing to eliminate moire. 

35 [0032] According to Claim 8 of the invention, in the image data retouching apparatus according to Claim 8 of the 
invention, the configuration is such that the smoothing is accomplished by a smoothing filter in which a matrix having a 
prescribed number of picture elements centering on the object picture elements is formed, and a prescribed coefficient 
is set in the position of each picture element in the matrix to approximately average the object picture elements. 
[0033] For moire picture elements as well, if determination and smoothing with the smoothing filter are applied to 

40 process picture elements in a prescribed range centering on the object picture elements, the determination of picture 
element characteristics and image processing will well match each other, resulting in efficient execution of smoothing. 
[0034] According to Claim 1 0 of the invention, in the image data retouching apparatus according to any of Claims 
1 through 9, the image data retouching unit obtains retouching values for the luminance of the image data, and 
retouches the image data by adding the retouching value of the tone value of each element color. 

45 [0035] If in the above-described configuration according to Claim 10 of the invention each picture element is com- 
posed of an element color, a retouching value for the computed luminance of each picture element is generated, and 
the luminance retouching value is added to the tone value of the element color of each element color to execute 
retouching of image data. Thus, retouching is carried out on the basis of a luminance value computed from each ele- 
ment color, and the resultant luminance-based retouched data are caused to reflect in the element colors. 

so [0036] Incidentally, the distribution of differences of tone levels may as well be figured out by utilizing the concept 
of vector value. For instance, differences of tone levels of the object picture element from surrounding picture elements 
in a prescribed range, which are first to be determined, and computed as vector values and, on the basis of the distri- 
bution of these vector values, the total number of surrounding picture elements whose vector values are greater and 
that of surrounding picture elements whose vector values are smaller are compared. Object picture elements sur- 

55 rounded by a relatively large number of picture elements whose vector values are greater are determined to be edge 
picture elements, and object picture elements surrounded by a relatively large number of picture elements whose vector 
values are smaller are determined to be moire picture elements. In executing this determining procedure, the picture 
element characteristic determining unit sequentially scans all the pictures. 
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[0037] After the determination of each picture element as to whether it is an edge picture element or a moire picture 
element is completed, the image data retouching unit subjects those picture elements determined to be edge picture 
elements to prescribed sharpening and those determined to be moire picture elements to prescribed smoothing. Simi- 
larly, the image data retouching unit, like the picture element characteristic determining unit sequentially scans all the 
5 picture elements to execute retouching of image data. Thus it determines each picture element to be an edge picture 
element or a moir6 picture element, and sharpens it if it is an edge picture element or smoothes it if it is a moire picture 
element. 

[0038] In this manner, determination of edge picture elements and moire picture elements and retouching of image 
data can be executed in an integrated process. 

10 [0039] On the other hand, the prescribed surrounding picture elements, in consideration of the range of likely 
impact on the picture elements to undergo determination or that of likely impact from the object picture elements, may 
be a picture element set constituting a 3*3 matrix, or one constituting a 5*5 matrix, around the object picture element, 
and the number of columns and rows making up the matrix may be altered as appropriate. Of course, the constitution 
of the range need not be limited to the above-described set of directly affecting or affected picture elements, but the 

75 range may be expanded or compressed as appropriate. 

[0040] Also, the mode in which the picture element characteristic determining unit determines the relative magni- 
tude of a vector value may use as the reference a prescribed threshold of the average of vector values. The reference 
may as well be the standard deviation, mode or median identified in figuring out the distribution of vector values, and 
may be altered as appropriate. Of course, the ranges of reference may be more than two, including a range where vec- 

20 tor values are judged to be greater, another where they are judged to be small, and still another range. 

[0041] The picture element characteristic determining unit may either determine the relative dominance of picture 
elements with greater or smaller vector values by simply comparing the numbers of picture elements with greater vector 
values and of ones with smaller vector values or the relative dominance of picture elements, divided into a greater 
number set and a smaller number set, whose cumulative total surpasses a prescribed threshold. 

25 [0042] The image data retouching unit may execute sharpening by either emphasizing, relative to surrounding pic- 
ture elements, the color components of picture elements determined to be edge picture elements or taking into account 
color components of surrounding picture elements and emphasizing the object picture elements relative to surrounding 
picture elements whose color components are taken into account in emphasizing the object picture elements. The latter 
case uses convolution by which the surrounding picture elements and the object picture element are multiplied by a pre- 

30 scribed coefficient and the multiples are added together. In such a case, the aforementioned prescribed matrix of sur- 
rounding picture elements is utilized. 

[0043] Similarly, the image data retouching unit may execute smoothing either by somewhat adjusting the intensi- 
ties of color components of what have been determined to be moire picture elements or by taking into account the color 
components of surrounding picture elements in the smoothing process, i.e. by also averaging the differences of tone 
35 levels from the surrounding picture elements. The latter case uses convolution by which the surrounding picture ele- 
ments and the object picture element are multiplied by a prescribed coefficient and the multiples are added together. In 
such a case, the aforementioned prescribed matrix of surrounding picture elements is utilized. 

[0044] It is sufficient for the image data acquisition unit to be able to acquire image data which express images in 
picture elements in a dot matrix form in multiple tones. Therefore, the input source of images may be a scanner, a digital 
40 camera or any device from which image data which express images in picture elements in a dot matrix form in multiple 
tones can be acquired, and the input source can be altered as appropriate. 

[0045] The images may be either monochrome or color. For color images, there is no particular limitation as to the 
setting of the coordinate system in chromatic space or the range of tones. 

[0046] Depending on how vector values based on differences of tone levels between the object picture element and 
45 picture elements surrounding them, there may be some picture elements which can be determined as neither edge pic- 
ture elements nor moire picture elements. Such picture elements differ little from surrounding picture elements in tone, 
it may sometimes obstruct accurate image data retouching to handle them as either edge picture elements or moire pic- 
ture elements. From this point of view, the configuration for determination may be such that picture elements of which 
a relatively large number have vector values smaller than those of what are determined to be moire picture elements 
so are determined to be intermediate picture elements, which are neither edge picture elements nor moire picture ele- 
ments. 

[0047] In such a configuration, the picture element characteristic determining unit determines to be intermediate 
picture elements those picture elements of which a relatively large number have vector values smaller than those of 
what are determined to be moire picture elements, which are neither edgeV»cture elements nor moire picture elements. 
55 Therefore, the picture element characteristic determining unit computes vector values, sets with respect to these vector 
values what leads to determination of edge picture elements, what leads to determination of moire picture elements and 
what leads to determination of intermediate picture elements, and handle those picture elements whose vector values 
are smaller than those corresponding to moire picture elements as intermediate picture elements. 
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[0048] As a specific example of configuration in which the picture element characteristic determining unit computes 
luminance values from differences of tone levels, the luminance of each of the picture elements is computed, and the 
differences of tone levels of these luminance values can be computed as the vector values. 

[0049] In such a configuration, the picture element characteristic determining unit computes by a prescribed for- 
5 mula luminance values from data on color components constituting image data acquired by the image data acquisition 
unit. On the basis of these luminance values, it computes differences of tone levels as vector values, and executes 
determination of edge picture elements, moire picture elements and intermediate picture elements. 
[0050] As another specific example of configuration in which the picture element characteristic determining unit 
computes luminance values from differences of tone levels, each picture element is separated into elements colors, and 
10 the difference of tone levels of each element color into which the picture element has been separated can be computed 
as the vector value. 

[0051] In such a configuration, the picture element characteristic determining unit separates each picture element 
into element colors, and computes the differences of tone levels of the separated element colors as the vector values. 
Here the element colors mean the color components constituting picture elements, i.e. R (red), G (green) and B (blue). 

75 When the characteristic of each picture element is to be determined from the vector value, what constitutes an edge 
picture element out of one element color may be determined to be an edge picture element, what constitutes an edge 
picture element out of two element colors may be determined to be an edge picture element, or only what constitutes 
an edge picture element out of three element colors may be determined to be an edge picture element, i.e. only what 
constitutes an edge picture element out of all the element colors may be determined to be an edge picture element. 

20 Therefore, this method of determination may be altered as appropriate. 

[0052] As one example of handling by the image data retouching unit of what have been determined by the picture 
element characteristic determining unit as intermediate picture elements, the configuration may be such that these 
intermediate picture elements are subjected to no image data retouching, either sharpening or smoothing. 
[0053] In such a configuration, the image data retouching unit executes no image data retouching, either sharpen- 

25 ing or smoothing on what have been determined by the picture element characteristic determining unit as intermediate 
picture elements. Thus, it holds the level of the picture elements corresponding to the image data acquired by the image 
data acquisition unit. 

[0054] Image data here often can provide various kinds of information, but the data to be used need not be fixed. 
For example, according to Claim 1 1 of the. invention, the configuration is such that, in the image data retouching appa- 
30 ratus according to any of Claims 1 through 10, in retouching image data of each picture element, an image data 
attribute specifying unit is caused to execute acquisition of attribute specification for the image data to be handled, and 
the picture element characteristic determining unit and the image data retouching unit are caused to execute their 
respective functions on the basis of the image data having the attribute acquired by the image data attribute specifying 
unit. 

35 [0055] According to the prior art, the attribute of the image data to be retouched is fixed. Namely, if the attribute is 
based on luminance signals, the luminance signals of the image data are retouched, or if it is based on RGB signals, 
the RGB signals of the image data are retouched. 

[0056] jn such a case, where the user desires an output of high quality, if image data are retouched on the basis of 
luminance signals, the output will be inferior to that of image data retouching based on RGB signals. Or where the user 
40 desires image data retouching in a short period of time, if image data are retouched on the basis of RGB signals, the 
execution will take a longer time than when image data are retouched on the basis of luminance signals. Thus, there is 
the problem that the fixation of the attribute may result in failure to satisfy the user's requirement. 
[0057] In the above-described configuration according to Claim 1 1 of the invention, the specification of the attribute 
of image data to be handled by the image data attribute specifying unit is acquired, and the picture element character- 
's istic determining unit and the image data retouching unit are caused to execute their respective functions on the basis 
of the image data having the attribute acquired by the image data attribute specifying unit. 

[0058] Thus, in processing image data retouching, instead of executing it on the basis of a prescribed attribute of 
image data, an attribute specified by the image data attribute specifying unit is selected, and the processing by the pic- 
ture element characteristic determining unit and the image data retouching unit is executed on the basis of this selec- 
so tion. 

[0059] Thus, the attribute of image data to base the execution of image data retouching on is determined in accord- 
ance with a prescribed specification, not only resulting in greater convenience of use but also making it possible to 
obtain an output satisfying the user's requirement. 

[0060] It is sufficient for the image data attribute specifying unit here to be able to specify the attribute on the basis 
55 of which image data are to be handled, and this attribute may be selected by the user or may be an attribute suitable 
for the composition of image data, identified by analysis of the image data. Or an attribute matching the state of the 
hardware resource to be applied may be specified. Of course, the choice of attribute is not limited to these, but if an 
attribute on the basis of which the image data retouching unit retouches image data can be specified, it can be altered 
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as appropriate. 

[0061] It can be any attribute that can be derived from picture elements expressed in multiple tones, which consti- 
tute image data. The applicable data may be binary data, RGT data, or luminance data computed from the RGB data. 
Of course, the choice of attribute is not limited to these, but if the attribute can express the state of image data, it can 

5 be altered as appropriate. 

[0062] As an example of attribute to be specified, according to. Claim 12 of the invention, in the image data retouch- 
ing apparatus according to Claim 11, the image data attribute specifying unit has a configuration to specify luminance 
signals as the attribute where high-speed image data retouching is desired and to specify an element color signal con- 
stituting images as the attribute where high-quality image data retouching is desired. 

io [0063] In the above-described configuration according to Claim 12, the image data attribute specifying unit, when 
specifying the attribute of image data to be retouched, specifies luminance signals as the attribute where high-speed 
processing is desired. Or where the quality of image data to be generated after image data retouching is required to be 
high, it specifies element color signals as the attribute, for instance RGB signals. 

[0064] Where the image data consist of RGB data, luminance signals can be computed on the basis of these RGB 
is data. If the number of picture elements constituting the image data is 100, since only one tone can be computed for 
each picture element on the basis of a luminance signal, there will be image data comprising 100 luminance tones. On 
the other hand, if RGB data are used, as each picture element has three colors, there will be image data comprising 
300 tones. Accordingly, there will be a difference in the number of data to undergo image data retouching, luminance 
signals will be specified where high-speed image data retouching is desired. Or where high-quality output is to be real- 
20 ized, element color signals constituting images, such as RGB signals will be specified as the attribute. 

[0065] This is due to the following reason. Usual luminance signals are not a mixture of RGB signals in equal pro- 
portions, but a mixture of 30% of R signals, 60% of G signals and 10% of B signals, all approximately. For this reason, 
if for instance B data have a large moire content, the moire content of the B data cannot be sufficiently reduced by 
retouching luminance signals and causing that retouching to be reflected in RGB signals. In order to sufficiently reduce 
25 the moire content of the B data then, each of the RGB data should be retouched. Therefore, if a high-quality output is . 
desired, RGB signals are specified. * ■ 

[0066] Incidentally, the image data retouching accomplished by the image data retouching unit may be any 
processing that retouches image data and thereby adjusts the quality of reproduced images. Therefore, it may be cho- 
sen out of appropriate alternatives including the sharpening and unsharpening of images. In such image data retouch- 
30 ing, a prescribed filter is often applied to each picture element. If it is desired to sharpen prescribed picture elements, 
a prescribed sharpening filter is applied to those picture elements to emphasize tones, or if it is desired to unsharpen 
prescribed picture elements, a prescribed smoothing filter is applied to uniform tones. 

[0067] It can be readily understood that this technique to determine the characteristic of each of picture elements 
constituting image data and retouching the picture elements according to their respective characteristics is not neces- 

35 sarily confined to a substantive apparatus, but can also function as a method for such processing. In this connection, 
according to Claim 13 of the invention, there is provided an image data retouching method for determining the charac- 
teristic of each picture element in image data which express images in a dot matrix form in multiple tones and retouch- 
ing each picture element in a way appropriate to its characteristic, the method comprising: an image data acquisition 
step to acquire the image data; a picture element characteristic determining step to figure out the distribution of differ- 

40 ences of tone levels between object picture elements, which are the picture elements of the image data acquired by the 
image data acquisition step, and neighboring picture elements in a prescribed range around the object picture ele- 
ments, and determining the characteristic. of each object picture element by comparing the distribution so figured out 
and a prescribed model distribution; and an image data retouching step to execute prescribed image processing 
according to the characteristics of picture elements determined by the picture element characteristic determining step. 

45 [0068] Thus it is certain that the technique is not necessarily confined to a substantive medium or the like, but also 
can be effective as a method. It goes without saying that the modes of carrying out the invention as disclosed in Claims 
1 through 12 can be applied to this image data retouching method in exactly the same way. 

[0069] Incidentally, the idea underlying the invention may be realized in many different modes of implementation. It 
may be either embodied in such an image data retouching apparatus by itself or used in a state of being built into some 

50 other hardware. It is therefore alternatively substantiated as software or hardware as appropriate. 

[0070] Where the idea underlying the invention is to be substantiated as software for an image data retouching 
apparatus, it can obviously be present on a recording medium on which the software is recorded and utilized as such. 
[0071] As an example of such alternative, according to Claim 14 of the invention, there is provided a medium on 
which is recorded an image data retouching program for determining the characteristic of each picture element in image 

55 data which express images in a dot matrix form in multiple tones and retouching each picture element in a way appro- 
priate to its characteristic, the program comprising: an image data acquisition function to acquire the image data; a pic- 
ture element characteristic determining function to figure out the distribution of differences of tone levels between object 
picture elements, which are the picture elements of the image data acquired by the image data acquisition function, and 
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neighboring picture elements in a prescribed range around the object picture elements, and determining the character- 
istic of each object picture element by comparing the distribution so figured out and a prescribed model distribution; and 
an image data retouching function to execute prescribed image processing according to the characteristics of picture 
elements determined by the picture element characteristic determining function. It goes without saying that the modes 
5 of carrying out the invention as disclosed in Claims 1 through 1 2 can be applied in exactly the same way to this medium 
on which the image data retouching program is recorded. 

[0072] Of course, such a recording medium may be a magnetic recording medium, an opto-magnetic recording 
medium or any other suitable recording medium that may be developed in the future. Exactly the same applies to any 
stage of copying, be it a primary copy or a secondary copy, with no room for doubt. 
w [0073] Further, even where the invention is realized with software in part and with hardware in part, there is no dif- 
ference whatsoever in the underlying idea. Or else, part of the program may be stored on a recording medium and read 
as required. 

[0074] Where the invention is to be realized with software, it can either take a configuration utilizing hardware 
and/or operating system, or can be realized separately from them. For instance, inputting of image data for interpolation 

75 can be achieved by causing an operating system to call a prescribed function or by inputting such data from hardware 
without calling such a function. Where the invention is to be realized with the intervention of an operating system actu- 
ally, in the process in which the program is recorded on a medium and circulated, it can be understood that the invention 
can be carried out with this program alone. Or, whereas what executes the program is a computer, the concept of com- 
puter here covers not only usual computers but may be anything else only if it is mounted with one or more CPUs and 

20 processes a program. For instance, obviously an intelligent printer or copying machine can be regarded as a computer 
in this sense. 

[0075] Further, where the invention is to be carried out with software, the invention is not only realized by a record- 
ing medium on which the pertinent program is recorded, but it is obvious for the invention to be realized as the program 
as such, and the program itself also is included in the coverage of the invention. 

25 

Brief Description of the Drawings 
[0076] 

30 Fig. 1 illustrates an image data retouching method in a preferred mode of carrying out the present invention with 

reference to claims. 

Fig. 2 schematically illustrates the appearance of a color copying machine to which this image data retouching 
method is applied for its substantiation. 

Fig. 3 schematically illustrates the configuration of the scanner of this color copying machine. 
35 Rg. 4 is a schematic block diagram illustrating the configuration of the server of this color copying machine; 
Fig. 5 is a schematic block diagram illustrating the configuration of this server. 
Fig. 6 schematically illustrates the configuration of the color printer of this color copying machine. 
Fig. 7 schematically illustrates the appearance of a computer system representing a variation of this color copying 
machine. 

40 Fig. 8 is a flowchart schematically showing the color copying executed by the copy server. 

Fig. 9 is a flowchart schematically showing the scanning executed by the copy server. 

Fig. 10 is a flowchart schematically showing the image data attribute selection executed by the copy server. 
Fig. 1 1 is a flowchart schematically showing the difference of tone levels computation executed by the copy server. 
Fig. 12 illustrates the process of preparation a distribution from differences of tone levels in the difference of tone 
45 levels computation. 

Fig. 1 3 is a flowchart schematically showing the picture element characteristic determination executed by the copy 
server. 

Fig. 14 illustrates an example of storage of picture element characteristics. 
Rg. 1 5 illustrates another example of storage of picture element characteristics. 
so Rg. 1 6 illustrates a specific example of edge picture element. 

Rg. 17 illustrates a specific example of moire picture element. 

Fig. 1 8 illustrates a specific example of intermediate picture element. 

Fig. 1 9 is a flowchart schematically showing the image data retouching executed by the copy server. 
Rg. 20 illustrates an example of sharpening filter. 
55 Rg. 21 illustrates an example of smoothing filter. 

Fig. 22 is a flowchart schematically showing the image data conversion executed by the copy server. 



BNSDOCID: <EP 1079601A1_ 



EP 1 079 601 AT 

Best Mode for Carrying Out the Invention 

[0077] Preferred embodiments of the present invention will be described with reference to the accompanying draw- 
ings. 

5 [0078] Fig. 1 illustrates an image data retouching method in a preferred mode of carrying out the invention with ref- 
erence to claims. 

[0079] Referring to Fig. 1 , a medium on which this image data retouching program is recorded acquires at an image 
data acquisition step Al image data which express images in a dot matrix form in multiple tones and at a picture ele- 
ment characteristic determining step A2 enters image data from the image data acquisition step Al to analyze the data 

w of the picture elements constituting the image data and to determine the characteristic of each picture element, i.e. 
whether it is an edge picture element, a moire picture element or an intermediate picture element. More specifically, 
first, with respect to object picture elements to be determined, differences of tone levels between those picture elements 
and surrounding picture elements in a prescribed range are computed as vector values and then, on the basis of the 
distribution of these vector values, the total number of picture elements whose vector values and that of picture ele- 

15 ments whose vector values are smaller are compared. The picture elements among which picture elements with greater 
vector values are relatively dominant are determined to be edgy picture elements, and those among with picture ele- 
ments with smaller vector values are relatively dominant are determined to be moire picture elements. 
[0080] Further, picture elements among with picture elements which have smaller vector values but cannot be 
determined to be moire picture elements are relatively dominant are determined to be intermediate picture elements. 

20 At the picture element characteristic determining step A2, all the picture elements are sequentially scanned to execute 
this determination procedure. When every picture element has gone through determination as to whether it is an edge 
picture element, a moire picture element or an intermediate picture element, at an image data retouching step A3, with 
respect to image data entered from the image data acquisition step A1 , prescribed sharpening is executed on picture 
elements which were determined as constituting edge picture elements, while prescribed sharpening is executed on 

25 picture elements which were determined as constituting moire picture elements. What were determined as constituting 
intermediate picture elements are subjected no processing at all. At the image data retouching step A3, as at the picture 
element characteristic determining step A2, all the picture elements are sequentially scanned to execute image data 
retouching. 

[0081] In carrying out such image data retouching, at an image data attribute specifying step A4 the attribute to be 
30 handled in the image data retouching, such as luminance signals or RGB signals, are specified. This may be specified 
either by the user or as dictated by the composition of image data. Therefore, if luminance signals are specified as the 
attribute, at the image data retouching step A3 image data retouching is executed according to the luminance data of 
each picture element or, if RGB signals are specified as the attribute, image data retouching is executed according to 
the RGB data of each picture element. 
35 [0082] Next, an external perspective view of a color copying machine to which this image data retouching method 
is applied for its substantiation is shown in Fig. 2. 

[0083] This color copying machine 1 0 consists of a color scanner 20, a copy server 30 and a color printer 40. As 
the color scanner 20 scans an image under the control of the copy server 30, the copy server 30 processes the image 
data read in by the scanning to generate print data and the color printer 40 performs printing on the basis of these print 
40. data. 

[0084] Fig. 3 schematically illustrates the configuration of the color scanner 20, which is a flat head type unit in this 
embodiment of the invention. 

[0085] Referring to the figure, underneath a transparent plate 21 on which the object of scanning is to be mounted 
there are supported an illumination lamp 22 and a line sensor 23 in a reciprocative sliding, and a drive belt 24a, pulleys 
45 24b and a drive motor 24c for driving them are arranged, all connected to a control circuit 25. When reading an image 
in, the illumination lamp 22 is turned on in accordance with a control signal from the control circuit 25 and, as it illumi- 
nates the object of scanning via the transparent plate 21, light reflected from the object of scanning irradiates the line 
sensor 23 via the transparent plate 21 . 

[0086] In the line sensor 23 here, one row of an RGB filter and a CCD element for each of three primary colors of 
50 light, or a total of three rows are usually arranged, and these three rows of CCD elements read in the color arrangement 
for one row of the object of scanning in the horizontal direction and output them as image data. On the other hand, the 
control circuit 25 moves these illumination lamp 22 and line sensor 24 integrally in the vertical direction of the object of 
scanning by driving the drive motor 24c; every time it moves them by a minute distance, acquires image data from the 
line sensor 23 and outputs them. This, while subjecting externally the object of scanning to main scanning in the hori- 
55 zontal direction, subjects it to subscanning in the vertical direction to generate two-dimensional image data. 

[0087] Figs. 4 and 5 are schematic diagrams illustrating the copy server 30. The copy server 30, schematically 
equivalent to a computer, has a configuration in which a RAM 33, a ROM 34, an operation panel 35, a hard disk 36 and 
an l/F 37 are connected to the bus 32 of a CPU 31 . 
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[0088] Here the color scanner 20 and the color printer 40 are connected via the l/F 37. In the ROM 34 are written 
a basic arithmetic program and a conversion table, and the CPU 31 , while using the RAM 33 as its work area, executes 
the arithmetic program, and references the conversion table as required. Although nothing particular is specified for the 
l/F 37 in this mode of carrying out the invention; it is sufficient for the l/F 37 to be able to connect the color scanner 20 
5 and the color printer 40 to the copy server 30. It may be connected by an LPT port or by a USB port or SCSI. 

[0089] The hard disk 36 is provided with a scanner driver 38a for driving the color scanner 20 and a printer driver 
38b for driving the color printer 40. The scanner driver 38a can input image data from the color scanner 20, and the 
printer driver 38b can output image data to the color printer 40. 

[0090] The hard disk 36 is also used as a buffer for temporarily storing these image data, stores an image data 
w retouching program 39 for reading in image data entered by the scanner driver 38a, determining the characteristics of 
the picture elements constituting the image data, carrying out prescribed image data retouching in accordance with this 
determination, outputting the retouched image data to the printer driver 38b and causing the color printer 40 to execute 
printing. 

[0091] In addition to these, the operation panel 35 is provided with various operation buttons including a scan start 
15 button 35a, a ten key block 35b for entering the number of copies to be printed, setting the items of image retouching, 
and setting the type of the object of scanning, whether it is a color image containing a photograph or the like or a mon- 
ochrome image such as a document, a liquid crystal display 35c for confirming operational information. The CPU 31 
can monitor the operation from the operation panel 35 via the bus 32. 

[0092] Fig. 6 schematically illustrates the configuration of the color printer40, which uses an ink jet system by which 

20 printing is accomplished by ejecting color inks in a dot matrix form onto recording paper. In more detail, it is configured 
of a print head 41 consisting of three print head units 41 a, a print head controller 42 for controlling this print head 41 , a 
print head shift motor 43 for moving the print head 41 in the column direction, a paper feed motor 44 for feeding the 
printing paper in the line direction, and a printer controller 45 serving as the interface for these print head controller 42, 
print head shift motor 43 and paper feed motor 44 with external devices. 

25 [0093] This color printer 40 uses inks of four colors for printing, and each of its print head unit 41 a is provided with 
two mutually independent rows of print nozzles. The color ink to be fed can be altered for each row of print nozzles. In 
the illustrated case, both rows of the print head unit 41 a to the left of the drawing are supplied with black ink (K), the left 
row of the print head unit 41 a to the right of the drawing is supplied with magenta color ink (M), the right row of the same 
with yellow ink (Y), the left row of the print head unit 41 a in the middle with cyanic ink (C), and the right row of the same 

30 is not used. * 

[0094] Incidentally, although inks of four colors are used in this embodiment, it is also possible to use inks of six 
colors by maximal utilization of the two rows of print nozzles in each of the three print head units 41 a. In this case, dark- 
colored and light-colored inks are used for cyanic and magenta colors, and yellow and black inks are added to them to 
use altogether six colors. Although in this mode of implementing the invention, this image data retouching apparatus is 

35 applied to a dedicated color copying machine 1 0 formed in an integrated manner having the copy server 30 as its core, 
needless to mention it is also possible to realize a color copying system with a personal computer 53 provided with a 
color scanner 51 and a color printer 52 as illustrated in Fig. 7. 

[0095] Fig. 8 is a flowchart showing some details of, out of possible procedures if color copying executed by the 
aforementioned copy server 30, image data retouching according to the present invention. 

40 [0096] Referring to the chart, the operator of the color copying machine 10, after placing the object of scanning on 
a flat bed 21 of the color scanner 20, presses down the scan start button 35a on the operation panel 35. This causes 
the color scanner 20 to start scanning in accordance with the above-described operational procedure. First, in order to 
generate image data consisting of picture elements of a prescribed resolution for the overall image of the flat bed 21 
including the object of scanning, the color scanner 20 executes scanning (step S100). Then, in retouching the gener- 

45 ated image data, it executes image data attribute selection to select the attribute of the image data to be retouched (step 
S150). Then, using the picture elements of the generated image data as the object picture elements, it executes differ- 
ence of tone levels computation on the basis of the selected attribute to compute differences of tone levels of the object 
picture elements from surrounding picture elements (step S200), and executes picture element characteristic determi- 
nation on the basis of these differences of tone levels to determine whether each picture element is an edge picture ele- 

50 ment, a moire picture element or an intermediate picture element (step S300). Next, the color scanner 20 executes 
image data retouching to sharpen the picture element if it is an edge picture element, smoothes the picture element if 
it is a moire picture element, or holds the original data of the picture element if it is an intermediate picture element (step 
S400). This image data retouching is also executed on the basis of the selected attribute. Upon completion of this image 
data retouching, the color scanner 20 executes image data conversion to generate print data to be fed to the color 

55 printer 40 (step S500), and the print data converted from the image data are transferred to the color printer 40, and 
printed out by this color printer 40. Thus, since the attribute of the image data to be retouched are selectable, if for 
instance image data retouching is desired to be executed quickly, luminance signals can be selected as the attribute, or 
if high picture quality is desired for the result of image data retouching, RGG signals can be selected as the attribute, 
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resulting in a desired image data retouching environment in either case. 

[0097] Next each of the processing steps S100 through S500 will be described in more specific detail with refer- 
ence to the flowcharts of Figs. 9 through 11, 13, 19 and 22. 
[0098] The flowchart of Fig. 9 shows scanning at step S100. 

5 [0099] As stated above, when the operator of this color copying machine places a document on the flat bed 21 as 
the object of scanning and presses down the scan start button 35a, an image reading instruction is issued to the color 
scanner 20 via the l/F 37 to start scanning (step S1 05). Then, the resolution of scanning either set by the operation with 
the ten key block 35b or preset in this color copying machine 10 is read out (step S1 10). Hereupon, the control circuit 
25 of the color scanner 20 turns on the illumination lamp 22, and starts image scanning by giving a drive instruction to 

10 the drive motor 24c to cause the illumination lamp 22 and the line sensor 23 to slide (step S1 15). Then the control circuit 
25, every time a prescribed distance is traveled, acquires image data read by the line. sensor 23, at the same time gen- 
erates image data which will take on a pre-designated resolution (step S120), and transmits them to the copy server 30. 
. On the part of the copy server 30, these image data are received via the l/F 37, and spooled on the hard disk 36. Here, 
when it is determined the scanning of the picture elements divided as described above has been wholly completed 

15 (step SI 25), the spooled image data are stored onto the hard disk 36 (step S130). Thus, the processing to scan the 
object of scanning at a prescribed resolution and to stored the image data as they are acquired constitutes the image 
data acquisition step A1 according to the invention. 

[0100] In this manner, the color scanner 20 generates the image data of the object to be scanned. These image 
data are subjected to various modes of retouching with a view to improving the picture quality or some other purpose. 
20 When carrying out such image data retouching, the image data should be handled in accordance with a prescribed 
attribute. Fig. 10 is a flowchart showing how the image data attribute selection at step S150 to select this attribute is 
processed. 

[0101] Referring to the chart, the attribute set by the user on the operation panel 35 of the copy server 30 is 
acquired (step S155). If this attribute is luminance (step S160), it is so set that, when image data are to be handled 
25 thereafter, prescribed processing be executed on luminance data (step S1 65). On the hand, if the acquired attribute is 
RGB, it is so set that, when image data are to be handled thereafter, prescribed processing be executed on RGB data 
(step S1 70). 

[0102] Fig. 1 1 is a flowchart showing how the difference of tone levels computation at step S200 is processed. 
[0103] Referring to the chart, image data on the image, which is the object of scanning generated by scanning are 

30 read out (step S205), and it is determined by the above-described image data attribute selection whether or not the 
selected attribute is luminance data (step S206). If it is luminance data, each of the picture elements constituting the 
image data is scanned and, while the scanned picture elements are made the object picture elements according to 
which the attribute of these picture elements is to be determined, the luminance of these object picture elements is 
computed (step S210). This luminance Y is computed by substituting the tones of the data on the element colors R, G 

35 and B, constituting the object picture elements, into the following Equation (1): 

Y= 0.30R + 0.59G + 0.1 IB (1) 

[0104] Next, the luminance of the picture elements surrounding these object picture elements, the surrounding pic- 
40 ture elements being formed in a 5*5 matrix form, is computed on the basis of Equation (1) (step S215). 

[01 05] On the other hand, where the selected attribute is RGB data, the RGB data on each of the object picture ele- 
ments are extracted (step S216), and at the same time the RGB data on each of the surrounding picture elements are 
extracted (step S217). 

[0106] Where the attribute is luminance data, the luminance tones of the object picture elements are subtracted 
45 from the luminance tone's of the surrounding picture elements are subtracted to compute the differences of tone levels. 
On the other hand, where the attribute is RGB data, for each of the R, G and B data the tones of the object picture ele- 
ments are subtracted from the tones of the surrounding picture elements to compute the differences of tone levels (step 
S220). Upon completion of computation of the differences of tone levels from the surrounding picture elements, a dis- 
tribution based on the frequency of picture elements contained in each of these differences of tone levels is prepared 
50 (step S225), and summation is done of each of the divisions determined to be edge picture elements, moire picture ele- 
ments and intermediate picture elements (step S230). 

[0107] By executing the picture element characteristic determination to be described later on, it is determined 
whether the object picture elements are edge picture elements, moire picture elements or intermediate picture ele- 
ments (step S235). It is determined whether or not determination of the characteristic of picture elements as the object 
55 picture elements by picture element characteristic determination has been executed on all the picture elements of 
image data read out at step S205 (step S240). If characteristic determination has been completed on all the picture ele- 
ments, difference of tone levels computation is ended or, if characteristic determination has not been completed on all 
the picture elements, the object picture elements are shifted, and the processing from step S210 onward is repeated. 
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[0108] Hereupon, the above-described difference of tone levels computation will be explained in further specific 
detail with reference to Fig. 12. Fig. 12A illustrates original image data in a 5*5 matrix. Each small box represents a pic- 
ture element. To position each picture element in (x, y) coordinates, the shaded picture element (0, 0) at the center con- 
stitutes an object picture element, and the picture element (-2, -2) through the picture element (2, 2) including this object 
5 picture element (0, 0) constitute the surrounding picture elements. 

[0109] The numeral in the box of each picture element represents the tone of the element. By the following Equa- 
tion (2), differences of tone levels between the object picture element and the surrounding picture elements are com- 
puted: 

10 b(n1, n2) = a(n1, n2) - a(N1, N2) (2) 

n1 = N1-2, N1-1, N1, N1 + 1, N1 + 2 
n2 = N2-2, N2-1 , N2, N2 + 1 , N2 + 2 

15 

where a(N1 , N2) is the tone of the object picture element, which is a(N1 , N2) = 64 in this mode of implementation; a(n1, 
n2) is the original tones of the surrounding picture elements, and b(n1 , n2), differences of tone levels between the sur- 
rounding picture elements and the object picture element. The result of this computation is shown in the 5*5 matrix of 
Fig. 12B. While preparing the distribution of these differences of tone levels, the distribution totaled for each section in 
20 Fig. 12C is generated. 

[0110] Although in this mode of implementing the invention, these sections are supposed to be section 1 [-255, - 
50), section 2 [-50, -10), section 3 [-10, 10) , section 4 [1 0, 50) and section 5 [50, 255], and the frequencies of differ- 
ences of tone levels are totaled forthese sections 1 through 5. In such a case, it is seen that, the respective totals being 
represented by C1 through C5, C1 = 0, C2 = 1 , C3 = 11 , C4 = 1 3 and C5 = 0. 
25 [01 1 1] Furthermore, although in this mode of implementation the sections of distribution are supposed to be section 
1 [-255, -50), section 2 [-50, -10), section 3 [-10, 10), section 4 [10, 50) and section 5 [50, 255], of course the way of 
section setting is not confined to this, but can be altered as appropriate. 

[0112] Processing at steps S225 through S235 is executed on a sole luminance datum for each picture element 
where the attribute is luminance data. On the other hand, where the attribute is RGB data, it is executed on the RGB 

30 data of each picture element. v 

[0113] Next will be described the picture element characteristic determination to be carried out at step S235 
explained above. Fig. 13 is a flowchart showing how this picture element characteristic determination is processed. 
[0114] Referring to the chart, first, C1 through C5 totaled for the object picture element are acquired (step S305). 
Then, it is determined whether not C1 through C5 satisfy the requirement for determination of an edge picture element 

35 represented by the following Equation (3) (step S310): 

CI > C2 > Te , C4 = C5 = 0 or (3) 
C5 > C4 > Te , C1 = C2 = 0 

40 

where Te is a predetermined threshold, Te = 5 in this mode of implementation. Or course, Te is not confined to 5, but 
can be altered as appropriate. 

[0115] If this Equation (3) is satisfied, the object picture element is determined to be an edge picture element (step 
S315), and a picture element characteristic flag indicating the characteristic of this picture element is supposed to be 

45 an edge flag (step S320). 

[0116] Since such an edge flag is the sole picture element characteristic flag for each picture element where the 
attribute is luminance data, it uses a configuration in which, as illustrated in Fig. 14A, the prescribed position of the bit 
datum representing the characteristic, provided for each object picture element, is turned on and off, and where the 
attribute is RGB data, as illustrated in Fig. 14B, the bit datum representing the characteristic, provided for the RGB data 

so of each object picture element, is turned on or off. 

[0117] As another example, a characteristic flag table manifesting the same configuration as the range of picture 
elements constituting the image data may be generated as shown in Fig. 15A, and data representing edge flags may 
be written into the respectively corresponding picture element positions. Of course, while Fig. 15A illustrates a case in 
which the attribute is luminance data, a characteristic flag table is generated for each of R, G and B as illustrated in Fig. 

55 15B where the attribute is RGB data. 

[01 18] Or if at step S3 1 0 the object picture element fails to satisfy the requirement for determination of an edge pic- 
ture element, it is determined whether or not the requirement for determination of a moire picture element represented 
by the following Equation (4) is satisfied (step S325): 
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C2 > C1 > Tm, C4= C5 = 0 or (4) 
C4 > C5 > Tm, C1 = C2 = 0 

5 where Tm is a predetermined threshold, Tm = 3 in this mode of implementation. Or course, Tm is not confined to 3, but 
can be altered as appropriate. 

[0119] If this Equation (4) is satisfied, theobject picture element is determined to be a moire picture element (step 
S330), and a picture element characteristic flag indicating the characteristic of this picture element is supposed to be a 
moire flag (step S335). 

w [0120] Such a moire flag may either turn on or off the prescribed position of the bit datum representing the tone 
datum of the object picture element as shown in Figs. 14A and 14B or Figs. 15A and 15B referred to above, or generate 
a characteristic flag table manifesting the same configuration as the range of picture elements constituting the image 
data and write a datum representing the moire flag in the corresponding picture element position. 
[0121] On the other hand a picture element that fails to satisfy the requirement for determination of a moire picture 

15 element in the determination at step S325 is determined to be an intermediate picture element (step S340), and a pic- 
ture element characteristic flag indicating the characteristic of this picture element is supposed to be an intermediate 
flag (step S345). Such an intermediate flag, too, may either turn on or off the prescribed position of the bit datum rep- 
resenting the tone datum of the object picture element as shown in Figs. 14A and 14B or Figs. 15A and 15B referred to 
above, or generate a characteristic flag table manifesting the same configuration as the range of picture elements con- 

20 stituting the image data and write a datum representing the intermediate flag in the corresponding picture element posi- 
tion. . 

[0122] If the selected attribute here is RGB data, it is necessary to determine for each of R, G and B whether or not 
procedures at steps S305 through S345 have been executed (step S350, S355). 

[0123] Hereupon, examples of configuration and distribution of edge picture elements, moire picture elements and 
25 intermediate picture elements are shown in Figs. 1 6 through 1 8. These Figs. 1 6 through 1 8 illustrate cases in which the 
attribute is luminance data. Where the attribute is RGB data, a distribution is generated for each of R r G and B. 
[0124] Fig. 16 illustrates a case in which object picture elements form edge picture elements. Fig. 16A shows orig- 
inal image data in a 5*5 matrix, in which each box represents a picture element. The shaded picture element (0, 0) con- 
- stitutes an object picture element, and the picture element (-2, -2) through the picture element (2, 2) including this object 
30 picture element (0, 0) constitute the surrounding picture elements. The numeral in the box of each picture element rep- 
resents the tone of the element. By Equation (2), differences of tone levels between the object v picture element and the 
surrounding picture elements are computed. The result of this computation is shown in the 5*5 matrix of Fig. 1 6B. While 
preparing the distribution of these differences of tone levels, the distribution totaled for each section in Fig. 1 6C is gen- 
erated. In such a case, it is seen that, the respective totals being represented by C1 through C5, C1 = 0, C2 = 0, C3 = 
35 1 , C4 = 7 and C5 = 17. As these satisfy the requirement of Equation (3) stated above, the object picture elements are 
determined to be edge picture elements. 

[0125] Further, Fig. 17 illustrates a case in which object picture elements form moire picture elements. Fig. 17A 
shows original image data in a 5*5 matrix, in which each box represents a picture element. The shaded picture element 
(0, 0) constitutes an object picture element, and the picture element (-2, -2) through the picture element (2, 2) including 

40 this object picture element (0, 0) constitute the surrounding picture elements. The numeral in the box of each picture 
element represents the tone of the element. By Equation (2), differences of tone levels between the object picture ele- 
ment and the surrounding picture elements are computed. The result of this computation is shown in the 5*5 matrix of 
Fig. 17B. While preparing the distribution of these differences of tone levels, the distribution totaled for each section in 
Fig. 17C is generated. In such a case, it is seen that, the respective totals being represented by C1 through C5, CI = 

45 0, C2 = 0, C3 = 5, C4 = 1 2 and C5 = 8. As these satisfy the requirement of Equation (4) stated above, the object picture 
elements are determined to be moire picture elements. 

[0126] Fig. 18 illustrates a case in which object picture elements form intermediate picture elements. Fig. 18A 
shows original image data in a 5*5 matrix, in which each box represents a picture element. The shaded picture element 
(0,0) constitutes an object picture element, and the picture element (-2, -2) through the picture element (2, 2) including 

50 this object picture element (0, 0) constitute the surrounding picture elements. The numeral in the box of each picture 
element represents the tone of the element. By Equation (2), differences of tone levels between the object picture ele- 
ment and the surrounding picture elements are computed. The result of this computation is shown in the 5*5 matrix of 
Fig. 18B. While preparing the distribution of these differences of tone levels, the distribution totaled for each section in 
Fig. 18C is generated. In such a case, it is seen that, the respective totals being represented by CI through C5, CI = 

55 0, C2 = 1 , C3 = 21 , C4 = 2 and C5 = 1 . As these satisfy the requirement of neither Equation (3) nor (4) stated above, 
the object picture elements are determined to be intermediate picture elements. 

[0127] Therefore, by selecting and specifying the attribute of data in retouching the image data of the object of 
scanning, generated by the scanning, the image data attribute selection constitutes the image data attribute specifying 
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step A4. Further by determining from the distribution of differences of tone levels from the surrounding picture elements, 
with respect to each of the picture elements constituting the acquired image data, whether the object picture elements 
are edge picture elements, moire picture elements or intermediate picture element, differences' of tone levels computa- 
tion and picture element determination constitutes the picture element characteristic determining step A2. 
5 [0128] Next, with reference to the flowchart of Fig. 19, how the image data retouching at step S400 is processed 
wilf be described. 

[0129] Referring to the chart, first, image data in which the characteristic flag of each picture element is stored are 
generated by the above-described scanning, and read out by picture element characteristic determination (step S405). 
Then the characteristic flag is confirmed for each of the picture elements constituting these image data. First it is deter- 

10 mined whether or not the characteristic flag is an edge flag (step S410) and, if it is an edge flag, a sharpening filter 
shown in Fig. 20 is applied to the pertinent picture elements (step S41 5). This sharpening filter is formed of a 3*3 matrix, 
wherein a large coefficient is set for the object picture element in the shaded box; to emphasize this object picture ele- 
ment, a negative coefficient is set for the surrounding picture elements immediately above, below and to the right and 
left of it; and a coefficient of zero is set for those surrounding picture elements not directly adjoining the object picture 

15 element. The application of the sharpening filter can be represented by the following Equation (6), where the tones of 
the original picture elements are represented by a(N1, N2), and those of the output picture elements after the applica- 
tion of the sharpening filter, by c(N1 , N2): 

20 c(/V1,A/2)= £ £ Mfcl.frS)* a(A/1 - *1,A/2- K2) (6) 

k1=1 k2=1 

where h(k1, k2) can be converted into the following Equations (7) and (8) in accordance with Fig. 20: 

25 

h(-1,-1) = 0, h(-1, 0) = -1,h(-1, 1) = 0, (7) 

h(0, -1) = -1, h(0, 0) = 5, h(0, 1) = -1, 

30 h(1,-1) = 0 l h(1,0) = -1,h(1,1)=0 

h(-1, -1) + h(-1, 0) + h(-1, 1) + h(0,-1) + h(0, 0) + h(0, 1) + h(1, -1)+ h(1, 0) + h(1, 1)= 1 (8) 

[0130] In this mode of implementation, a configuration in which a sharpening filter in a 3*3 matrix form is applied to 
35 emphasize the edge picture elements is used, but of course the sharpening filter is not confined to one in a 3*3 matrix 
form, and one in a 5*5 matrix form can also be used if Equation (8), i.e. the condition that the total of the coefficients in 
all the boxes be 1 , is satisfied, permitting alteration as appropriate. Then, picture elements to which the sharpening filter 
has been applied are stored as retouched picture elements (step S420). 

[0131] Or if the characteristic flag is determined not to be an edge flag at step S410, it is determined whether or not 
40 the characteristic flag is a moire flag (step S425). If it is a moire flag, a smoothing filter illustrated in Fig. 21 is applied to 
the pertinent picture elements (step S430). This smoothing filter is formed of a 3*3 matrix, wherein the same coefficient 
is set for the object picture element in the shaded box and the surrounding picture elements to average the tone of this 
object picture element with that of the surrounding picture elements. The application of the smoothing filter can be rep- 
resented by the following Equation (9), where the tones of the original picture elements are represented by a(N1, N2), 
45 and those of the output picture elements after the application of the smoothing filter, by c(N1, N2): 

c(A/1,W2)= £ £ g(*l,fr2)* a(AM - *l f A/2 - K2) (9) 

k1=1 k2=1 

50 

where g(k1, k2) can be converted into the following Equations (10) and (1 1) in accordance with Fig. 21 : 

g(-1, -1) = l/9, g(-l,0)= 1/9, g(-1, 1)= 1/9, (10) 

55 

g(0, -1) = 1/9, g(0, 0) = 1/9, g(0, 1) = 1/9, 
g(1,-l)= 1/9, g(1,0)= 1/9, g(1,l) = 1/9 
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g(-1 , -1 ) + g(-1 , 0) + g(-1 . 1 ) + g(0,-1 ) + g(0, 0) + g(0, 1 ) + g(1 , -1 ) + g(1 , 0) + g(l , 1 ) = 1 



[0132] In this mode of implementation, a configuration in which a smoothing filter in a 3*3 matrix form is applied to 
average the moire picture elements is used, but of course the smoothing filter is not confined to one in a 3*3 matrix form, 

5 and one in a 5*5 matrix form* can also be used if Equation (1 1 ), i.e. the condition that the total of the coefficients in all 
the boxes be 1 , is satisfied, or the coefficients may be weighted as appropriate. For instance, a smaller coefficient may 
be set for picture elements far from the object picture element, and a large one, for picture elements close to it. The pic- 
ture elements to which the smoothing filter has been applied are stored as retouched picture elements (step S420). 
[0133] On the other hand, if the characteristic flag is found not to be a moire flag at step S425, the pertinent picture 

w elements are determined to be intermediate picture elements, and neither the sharpening filter nor the smoothing filter 
is applied to them (step S435). Thus the tones of the original picture elements are held at the tones of the output picture 
elements. These intermediate picture elements, too, are stored as retouched picture elements (step S420). 
[0134] In executing the image data retouching here, if the selected attribute is RGB, image data retouching needs 
to be applied to each of R, G and B (step S440, S445). The determination so far described is applied to all the picture 

15 elements including edge picture elements, moire picture elements and intermediate picture elements (step S450), and 
image data resulting from the application of either the sharpening filter or the smoothing filter to each of the picture ele- 
ments constituting the original image data are generated (step S455). 

[01 35] In executing image data retouching involving the application of either filter on the basis of the luminance data 
of the picture elements here, the tones of the RGB data after the image data retouching are represented by R', G' and 
20 B\ respectively, and those of the RGB data before the retouching, by R, G and B, respectively If the luminance after the 
application of the sharpening filter or the smoothing filter to luminance Y is represented by luminance Y', R' f G' and B' 
can be computed by the following Equation (12): 

AY = Y' - Y (12) 

25 

R' = R + AY 
G' = G + AY 

30 B' = B + AY 

[01 36] On the other hand, where either filter is applied on the basis of the RG B data of the picture elements, as the 
filter is directly applied to the RGB data, the R', G' and B' resulting from the multiplication by the filter will become, as 
they are, the retouched image data. 
35 [0137] Since a prescribed fitter is applied to the image data acquired by scanning to retouch the image data on the 
basis of the picture element characteristic determination, the image data retouching constitutes the image data retouch- 
ing step A3. 

[0138] The image data having gone through retouching are subjected to color conversion and other processing, 
and delivered to the color printer 40 to execute printing. 
40 [0139] Hereupon, the flowchart of Fig. 22 shows how the image data conversion is executed on the retouched 
image data before they are delivered to the color printer 40. 

[0140] Referring to the chart, first the image data retouching is executed, arid the image data in which the picture 
elements are configured according to the output one are entered (step S505) and subjected to color conversion (step 
S510). If the image data are in the usual RGB 256 tones, the printer 50 will need color image data for printing in CMYK 
45 2 tones, and accordingly color conversion and tone conversion are required. Therefore at step 510, color image data in 
RGB 256 tones are converted into color image data in CMYK 256 tones. This color conversion may be executed by uti- 
lizing LUT in accordance with a standard technique. Then the CMYK 256 tones are converted into halftones of CMYK 
2 tones (step S515). The halftone print data are transmitted from the personal computer 10 to the printer 50 by parallel 
communication (step S520). 

so [0141] Then the color printer 40 enters these print data and executes printing on a print medium. Incidentally, 
although the positioning of various programs was described along the flow of processing, in practice they are realized 
in many different modes since a plurality of devices execute them at coordinated timings. Each program may be either 
executable independent of others or part of some other program. It is also possible to replace processing by software 
with processing by hardware, and in this context the term "module" refers to common means for both software and hard- 

55 ware. 

[0142] In this way, it is made possible to acquire image data from' an image input device such as the color scanner 
20, compute with respect to each of the picture elements constituting these image data differences of tone levels from 
the surrounding picture elements, prepare a distribution of these differences of tone levels, and determine on the basis 
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of the distribution whether each picture element is an edge picture element, a moire picture element or an intermediate 
picture element. Then, in accordance with the determination, the image data are retouched by applying a sharpening 
filter if they are edge picture elements, or a smoothing filter if they are moire picture elements, or the tones of the original 
image are held if they are intermediate picture elements, making it possible to carry out the whole execution from deter- 
5 mi nation to retouching in a serial process. This facilitates simplification of the flow of processing from determination to 
image data retouching, and at the same time speeding up the processing. Thus the user of the color copying machine 
10 is enabled to acquire prints of appropriately retouched images at higher speed. 

[0143] Furthermore, where luminance data are used as the attribute, after computing the luminance from the RGB 
data of the picture elements, each step of processing is executed on this luminance, resulting in a simplified configura- 
te tion of processing and faster image data retouching. Or where RGB data are used as the attribute, each step of 
processing is executed on the RGB data of the picture elements, resulting in a more elaborate configuration of process- 
ing and accordingly slower image data retouching. On the other hand, as each of the RGB data is reflected in the 
retouching, the quality of retouched image data is enhanced. Therefore the user can select luminance data in the image 
data attribute selection at step S150 if high-speed image data retouching is desired, or can select RGB data if a high 
15 quality image output desired, so that the image data retouching can be accomplished in the desired way according to 
the occasion. 

Industrial Applicability 

20 [0144] As hitherto described, since according to the present invention the distribution of differences of tone levels 
from neighboring. picture elements is figured out and the characteristics of the picture elements are determined accord- 
ing to the pattern of that distribution, it can provide amediumon which is recorded an image data retouching program 
which involves no need for altering the way of determination for each characteristic and, while facilitating determination, 
increase the speed of process by tightening the linkage to image processing. 

25 [0145] According to Claim 2 of the invention, determination can utilize a relatively simple way of summation, merely 
discriminating the distribution of relatively small differences of tone levels and that of relatively large differences, result- 
ing in a simplified configuration. 

[0146] According to Claim 3 of the invention, determination can utilize a relatively simple way of summation, that is 
polarization between positive or negative polarization of distribution, resulting in a simplified configuration. 
30 [0147] According to Claim 4 of the invention, edges can be readily determined by combining the distribution of large 
differences of tone levels and positive or negative polarization. 

[0148] According to Claim 5 of the invention, suitable processing for edge picture elements can be accomplished 
by sharpening the edge picture elements. 

[0149] According to Claim 6 of the invention, since picture elements in a prescribed range around object picture ele- 
35 ments are focused on, as in determination, for sharpening with a sharpening filter, resulting in relatively well coordinated 
determination and image processing. 

[0150] According to Claim 7 of the invention, moire can be readily determined by combining the distribution of small 
differences of tone levels and positive or negative polarization^ 

[0151] According to Claim 8 of the invention, suitable processing for moire picture elements can be accomplished 
40 by smoothing the moire picture elements. 

[0152] According to Claim 9 of the invention, since picture elements in a prescribed range around object picture ele- 
ments are focused on, as in determination, for smoothing with a smoothing filter, resulting in relatively well coordinated 
determination and image processing. 

[0153] According to Claim 1 0 of the invention, there is provided an image data retouching method whereby the con- 
45 figuration of processing for image data retouching can not only be simplified but also increased in speed through col- 
lective processing of determination of edge picture elements and moire picture elements and image data retouching on 
these picture elements. 

[0154] According to Claim 1 1 of the invention, there is provided an image data retouching method whereby the 
attribute of image data to be retouched in accordance with a prescribed procedure is determined, resulting in greater 
so convenience of use and enabling the user to obtain the output in a more desirable way. 

[0155] According to Claim 12 of the invention, specific attributes in accordance with which image data are to be 
handled can be presented. 

[0156] According to Claim 1 3 of the invention, there is provided an image data retouching apparatus providing sim- 
ilar benefits, and according to Claim 14 of the invention, there is provided a similar image data retouching method. 



55 



Claims 

1 . An image data retouching apparatus for determining the characteristic of each picture element in image data which 
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express images in a dot matrix form in multiple tones and retouching each picture element in a way appropriate to 
its characteristic, said apparatus comprising: 

an image data acquisition unit for acquiring said image data; 

5 a picture element characteristic determining unit for figuring out the distribution of differences of tone levels 

between object picture elements, which are the picture elements of the image data acquired by said image 
data acquisition unit, and neighboring picture elements in a prescribed range around the object picture ele- 
ments, and determining the characteristic of each object picture element by comparing the distribution so fig- 
ured out with a prescribed model distribution; and 

10 an image data retouching unit for executing prescribed image processing according to the characteristic of pic- 

ture elements determined by said picture element characteristic determining unit. 

2. An image data retouching apparatus, as claimed in Claim 1 ; wherein said picture element characteristic determin- 
ing unit utilizes for determination the distribution in a range of smaller differences and the distribution in a range of 

is greater differences. 

3. An image data retouching apparatus, as claimed in Claim 2, wherein said picture element characteristic determin- 
ing unit utilizes for determination the positive or negative polarization of said distribution. 

20 4. An image data retouching apparatus, as claimed in Claim 3, wherein said picture element characteristic determin- 
ing unit determines picture elements to be edge picture elements if the distribution in said range of greater differ- 
ences is dominant and if said distribution is polarized positively or negatively. 

5. An image data retouching apparatus, as claimed in Claim 4, wherein said image data retouching unit executes 
25 sharpening of images if said object picture elements are determined to be edge picture elements. 

6. An image data retouching apparatus, as claimed in Claim 5, wherein said sharpening forms a matrix having a pre- 
scribed number of picture elements centering on an object picture element and is executed by a sharpening filter 
wherein a prescribed coefficient to emphasize the object picture element is set in each picture element position in 

30 the matrix. 

7. An image data retouching apparatus, as claimed in Claim 3 or 4, wherein said picture element characteristic deter- 
mining unit determines picture elements to be moire picture elements if the distribution in said range of smaller dif- 
ferences is dominant and if said distribution is polarized positively or negatively. 

35 

8. An image data retouching apparatus, as claimed in Claim 7, wherein said image data retouching unit executes 
smoothing of images if said object picture elements are determined to be moire picture elements. 

9. An image data retouching apparatus, as claimed in Claim 8, wherein said smoothing forms a matrix having a pre- 
40 scribed number of picture elements centering on an object picture element and is executed by a smoothing filter 

wherein prescribed coefficients are set to roughly average said object picture elements in different picture element 
positions in the matrix. 

10. An image data retouching apparatus, as claimed in any of Claims 1 through 9, wherein said image data retouching 
45 unit obtains a retouching value for the luminance value of said image data, and adds the retouching value to the 

tone values of element colors to retouch the image data. 

11. An image data retouching apparatus, as claimed in any of Claims 1 through 10, wherein: 

so in retouching the image data of the picture elements, an image data attribute specifying unit for acquiring spec- 

ification of the attribute of image data to be handled is caused to execute the function thereof, and 
said image data retouching unit is caused to execute the function thereof on the basis of the image data having 
the attribute acquired by said image data attribute specifying unit. 

55 12. An image data retouching apparatus, as claimed in any of Claims 1 1 , wherein said image data attribute specifying 
unit specifies luminance signals as the attribute when high-speed image data retouching is desired and specifies 
element color signals constituting an image as the attribute when high-quality image data retouching is desired. 
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13. An image data retouching method for determining the characteristic of each picture element in image data which 
express images in a dot matrix form in multiple tones and retouching each picture element in a way appropriate to 
its characteristic, said method comprising: 

an image data acquisition step to acquire said image data; 

a picture element characteristic determining step to figure out the distribution of difference of tone levels 
between object picture elements, which are the picture elements of the image data acquired by said image 
data acquisition step, and neighboring picture elements in a prescribed range around the object picture ele- 
ments, and determining the characteristic of each object picture element by comparing the distribution so fig- 
ured out and a prescribed model distribution; and 

an image data retouching step to execute prescribed image processing according to the characteristic of pic- 
ture elements determined by said picture element characteristic determining step. 

14. A medium on which is recorded an image data retouching program for determining the characteristic of each pie- 
rs ture element in image data which express images in a dot matrix form in multiple tones and retouching each picture 

element in a way appropriate to its characteristics, said program comprising: 

an image data acquisition function to acquire the image data; 

a picture element characteristic determining function to figure out the distribution of differences of tone levels 
20 between object picture elements, which are the picture elements of the. image data acquired by said image 

data acquisition function, and neighboring picture elements in a prescribed range around the object picture ele- 
ments, and determine the characteristic of each object picture element by comparing the distribution so figured 
out and a prescribed model distribution; and* 

an image data retouching function to execute prescribed image processing according to the characteristic of 
25 picture elements determined by said picture element characteristic determining function. 
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FIG.16A 
Original image 
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FIG.17A 
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FIG.18A 
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FIG.19 
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